
TITLE 

Method for processing audio-signals. 

ARE A OF THE INVENTION 
5 The invention is related to the area of speech enhancement of audio signals, and more 
specifically to a method for processing audio signal in order to enhance speech 
components of the signal whenever they are present. Such methods are particularly 
applicable to hearing aids, where they allow the hearing impaired person to better 
communicate with other people. 

10 

BACKGROUND OF THE INVENTION 

The problem of extracting a signal of interest from noisy observations is well known by 
acoustics engineers. Especially, users of portable speech processing systems often 

15 encoimter the problem of interfering noise reducing the quality and intelligibility of 
speech. To reduce these harmful noise contributions, several single channel speech 
enhancement algorithms have been developed [1-4]. Nonetheless, even though single- 
channel algorithms are able to improve signal quality, recent studies have reported that 
they are still unable to improve speech intelligibihty [5]. In contrast, multiple- 

20 microphone noise reduction schemes have been shown repeatedly to increase speech 
intelligibility and quality [6,7] . 

Multiple microphone speech enhancement algorithms can be rougjily classified into 
quasi-stationary spatial filtering and time-variant envelope filtering [8]. Quasi-stationary 

25 spatial filtering exploits the spatial configuration of the sound sources to reduce noise by 
spatial filter. The filter characteristics do not change with tlie dynamics of speech but 
with the slower changes in the spatial configuration of the sound sources. They achieve 
almost artefact-fiee speech enhancement in simple, low reverberating environments and 
computer simulations. Typical examples are adaptive noise cancelling, positive and 

30 difTerential beam-forming [30] and blind source separation [28,29]. The most promising 
algorithms of this class proposed hitherto are based on blind source separation (BSS). 
BSS is the sole technique, which aims to estimate an exact model of the acoustic 
environment and to possibly invert it. It includes the model for de-mixing of a number of 



CONFIRMATION COPY 



wo 2005/020633 PCT/EP2004/009283 

2 

acoustic sources from an equal number of spatially diverse recordings. Additionally, 

midti-path propagation, though reverberation is also included in BSS models. 

The basic problem of BSS consists in recovering hidden source signals using only its 

linear mixtures and nothing else. Assume statistically independent sources 
s(^) [s^ (tX . . . , (0] These sources are convolved and nuxed in a linear medium 
leading to ^' sensor signals ^^'^ t^(0»-,^rf, (01 ^^^y ^^^i^^^ additional noise: 
x(0 = EG(T)s(f-r) + n(0. 

(1) 

The aim of source separation is to identify the multiple channel transfer characteristics 
G(t), to possibly invert it and to obtain estimates of the hidden sources given by: 



u(o=i: 



W(r)x(^-r) 

(2) 

where W(t) is the estimated inverse multiple channel transfer characteristics of G(t). 
Numerous algorithms have been proposed for the estimation of the inverse model W(t;). 
They are mainly based on the exploitation of the assumption on the statistical 
independence of the hidden source signal. The statistical independence can be exploited 
in different ways and additional constraints can be introduced, such as for example 
intrinsic correlations or non-stationnarity of source signals and/or noise. As a result a 
large number of BSS algorithms under various implementation forms (e.g. time domain, 
frequency domain and time-frequency domain) have been proposed recently for 
multiple-channel speech enhancement (see for example [28,29]). 



Dogan and Stems [9] use cumulant based source separation to enhance the signal of 
interest in binaural hearing aids. Rosea et al. [10] apply blind source separation for de- 
mixing delayed and convoluted soiirces from the signals of a microphone array. A post- 
processing is proposed to improve the enhancement. Jourjine et al. [1 1] use the statistical 
distribution of the signals (estimated using histograms) to separate speech and noise. 
Balan et al. [2] propose an autoregressive (AR) modelling to separate sources from a 
degenerated mixture. Several approaches use the spatial information given by a plxirality 
of microphone using beamformers. Koroljow and Gibian [12] use first and second order 
beamformer to adapt the directivity of the hearing aids to the noise conditions. 
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Bhadkamkar and Ngo [3] combine a negative beamformer to extract tiie speech source 
and a post-processing to remove the reverberation and echoes. Lindemann [13] uses a 
beamformer to extract the energy from the speech source and an omni-directional 
microphone to obtain the whole energy from the speech and noise sources. The ratio 
5 between these two energies allows to enhance the speech signal by a spectral weighting. 
Feng et al. [14] reconstructs the enhanced signal using delayed versions of the signals of 
a binaural hearing aid system. 

BSS techniques have been shown to achieve almost artefact-free speech enhancement in 
10 simple, low reverberating environments, laboratory studies and computer simulations but 
perform poorly for recordings in reverberant environment or/and with difJuse noise. One 
could speculate that in reverberant environments the number of model parameters 
becomes too large to be identified accurately in noisy, non-stationary conditions. 

In contrast, envelope filtering (e.g. Wiener, DCT-Bark, coherence and directional 
filtering) do not yield such failures since they use a simple statistical description of the 
acoustical enviromnent or the binaural interaction in the human auditory system [8]. 
Such algorithms process the signal in an appropriate dual domain. The envelope of the 
target signal or equivalently a short time weighting index (short-time signal-to-noise 
ratio (SNR), coherence) is estimated in several frequency bands. The target is assumed to 
be of frontal incidence and the enhanced signal is obtained by modulating the spectral 
envelope of the noisy signal by the estimated short time weighting index. The adaptation 
of the weighting index has a temporal resolution of about the syllable rate. Dual channel 
approaches based on the statistical description of the sources using the coherence 
function have been presented [1,15-17]. Further improvements have been obtained by 
merging spatial coherence of noisy sound fields, masking properties of the human 
auditory system and subspace approaches [19]. 

Multi-channel speech enhancement algorithms based on envelope filtering are 
30 particularly appropriate for complex acoustic environments, namely diffuse noise and 
highly reverberating. Nevertheless, they are imable to provide loss-less or artefact-free 
enhancement. Globally, they reduce noise contributions in the time-frequency domains 
without any speech contributions. In contrast, in time-frequency domains with speech 
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contributions, the noise cannot be reduced and distortions can be introduced. This is 
mainly the reason why envelope filtering might help reducing the listening efifort in noisy 
environments but intelligibility improvement is generally leaking [20]. 

5 The above considerations point out that performance of multiple chaimel speech 

enhancement algorithms depend essentially on the complexity of the acoustical context. 
A given algorithm is appropriated for a specific acoustic environment and in order to 
cope with changing properties of the acoustic environment composite algorithms have 
been proposed more recently. 

10 

The approach proposed by Melanson and Lindemann in [21] consists in a manual 
switching between different algorithms to enhance speech under various conditions. A 
manual switching between several combinations of filtering and dynamic compression 
has also been proposed by Lindemann et al. [22]. 

15 

More advanced techniques using an automatic switching according to different noise 
conditions have been proposed by Killion et al. in [23]. The input of the hearing aid is 
switched automatically between omnidirectional and directional microphone. 

20 A strategy selective algorithm has been described by Wittkop [24]. This algorithm uses 
an envelope filtering based on a generaUzed Wiener approach and an envelope filtering 
invoking directional inter-aural level and phase differences. A coherence measure is used 
to identify the acoustical situations and gradually switch off the directional filtering with 
increasing complexity. It is pointed out that this algorithm helps reducing the hstening 

25 effort in noisy environments but that intelligibility improvement is still lacking. 

Therefore, it is the aim of the present invention to provide a composite method including 
source separation and coherence based envelope filtering. Source separation and 
coherence based envelope filtering are achieved in the time Bark domain, i.e. in specific 
30 fi-equency bands. Source separation is performed in bands where coherent soimd fields of 
the signal of interest or of a predominant noise source are detected. Coherence based 
envelope filtering acts in bands where the sound fields are diffiase and /or where the 
complexity of the acoustic environment is too large. Source separation and coherence 
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based envelope filtering may act in parallel and are activated in a smooth way through a 
coherence measure in the Bark bands. 



It is further an issue of the present invention to provide a real binaural enhancement of 
5 the observed sound field by using the multiple channel transfer characteristics identified 
by source separation. Indeed, commonly speech enhancement algorithms achieve 
mainly a monaural speech enhancement, which implies that users of such devices loose 
the ability to localize sources. A promising solution, which could achieve real binaural 
speech enhancement, consists of a device with one or two microphones in each ear and 

10 an RF-link in-between. The benefit for the user would be enormous. Notably it has been 
reported that binaural hearing increases the loudness and signal-to-noise ratio of the 
perceived sound, it improves inteUigibihty and quality of speech and allows the 
localization of sources, which is of prime importance in situations of danger. 
Lindemann and Melanson [25] propose a system with wireless transmission between the 

15 hearing aids and a processing unit wearied at the belt of the user. Brander [7] similarly 
proposes a direct communication between the two ear devices. Goldberg et al. [26] 
combine the transmission and the enhancement. Finally optical transmission via glasses 
has been proposed by Martin [27], Nevertheless in none of these approaches a virtual 
reconstruction of the binaural sound filed has been proposed. The approach proposed 

20 herein, namely exploitation of the multiple channel transfer characteristics identified by 
source separation to reconstruct the real soxmd field and attenuat noise contribution 
considerably improve the security and the comfort of the listener. 
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SUIVIMARY OF THE BSTVENTION 

45 

The invention comprises a method for processing audio-signals whereby audio signals 
are captured at two spaced apart locations and subject to a transformation in the 
perceptual domain (Bark or Mel decomposition), whereupon the enhancement of the 
speech signal is based on the combination of parametric (model based) and non- 
50 parametric (statistical) speech enhancement approaches: 
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a. a soxirce separation process is performed to give a first estimate of the 
wanted signal parts and the noise parts of the microphone signals and 

b. a coherence based envelope filtering is performed to give a second 
estimate of the wanted signal parts of the microphone signals, 

5 and where further a sound field difiuseness detection is performed on the at least two 
signals, whereby fiirther the sound field difiuseness detections is used to mix the output 
fi'om the first and the second source separation process in order to achieve the best 
possible signal The transfer functions estimated by the source separation algorithms are 
used to reconstruct a virtual stereophonic sound field (spatial localisation of the different 
1 0 sound sources). 

When the speech and noise sowces are in the direct sound field (direct path between 
sound sources and microphones is dominant, reverberation is low), the transmission 
transfer function firom each source in each source ear system can be estimated and used 

15 to separate speech and noise signals by the use of source separation. These transfer 
functions are estimated using source separation algorithms. The learning of the 
coefficients of the transfer functions can be either supervised (when only the noise 
source is active) or blind (when speech and noise sources are active simultaneously). The 
learning rate in each firequency band can be dependant on the signals characteristics. The 

20 signal obtained with this approach is the first estimated of the clean speech signal. 

When the noise signal is in the reverberant sound field (contributions firom 
reverberations is comparable to those of the direct path), soiirce separation approaches 
fails due to the complexity of the transfer functions to be evaluated. A statistical based 

25 envelope filtering can be used to extract speech from noise. The short-time coherence 
function calculated in the transform domain (Bark or Mel) allows estimating a 
probabiHty of presence of speech in each Bark or Mel firequency band. Applying it to the 
noisy speech signal allows to extract the bands where speech is dominant and attenuate 
those where noise is dominant. The signal obtained with this approach is the second 

30 estimate of the clean speech signal. 

These two estimates of the clean speech signal are then mixed to optimise the 
performance of the enhancement. The mixing is performed independently in each 
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frequency band, depending on the sound field characteristic of each frequency band. The 
respective weight for each approach and for each frequency band is calculated from the 
coherence function. 



5 During the combination of the signals calculated from the two approaches, the transfer 
ftinctions estimated by source separation are used to reconstruct a virtual stereophonic 
soimd field and to recover the spatial information from the different sources. 

In a fiirther embodiment of the invention the soxmd field diffuseness detection is based 
10 on the value of a short-time coherence ftinction where the coherence fimction is 
expressed as: 



This function varies between zero and one, according to the amoimt of "coherent" signal 
15 When the speech signal dominates the frequency band, the coherence is close to one and 
when there is no speech in the frequency band, the coherence is close to zero. Once the 
diffuseness of the soxmd field is Icnown, the results of the source separation and of the 
coherence based approach can be combined optimally to enhance the speech signals. The 
combination can be the use of one of the approach when the noise source is totally in the 
20 direct soxmd field or totally in the diffuse soxmd field, or a combination of the resxilts 
when some of the frequency bands are in the direct soxmd field and other are in the 
diffuse soxmd field. 



25 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of the proposed approach. 
Fig. 2 is a complete mixing model for speech and noise sources. 
Fig. 3 is a modified mixing model. 
30 Fig. 4 is a De-mixing model. 
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DESCRIPTION OF A PREFERRED EMBODIMENT 

The aim of a hearing aid system is to improve the inteUigibihty of speech for hearing- 
impaired persons. Therefore it is important to take into accoimt the specificity of the 
5 speech signal. Psycho-acoustical studies have shown that the human perception of 

frequency is not linear with frequency but the sensitivity to frequency changes decreases 
as the frequency of the sound increases. This property of the human hearing system has 
been widely used in speech enhancement and speech recognition system to improve the 
performances of such systems. The use of critical band modeling (Bark or Mel 
10 frequency scale) allows to improve the statistical estimation of the speech and noise 
characteristics and, thus, to improve the quality of the speech enhancement. 

When the speech and noise sources are in the direct sound field (low reverberating 
acoustical environment), the transmission transfer function of each source in each ear 
15 system can be estimated and used to separate the speech and noise signals. The mixing 
system is presented in figure 2. 

The mixing model of figure 2 can be modified to be equivalent to the model of figure 3. 

20 The inversion of the transfer fimctions HI 2 and H21 allows recovering the original 
signals up to the modification induced by the transfer function Gl 1 and G22. The de- 
mixing model is presented in figure 4. 

The de-mixing transfer fimctions W12 and W21 can be estimated using higher order 
25 statistics or time delayed estimation of the cross-correlation between the two. The 

estimation of the model parameters can be either supervised (when only one source is 
active) or blind (when the speech and noise sources are active simultaneously). The 
learning rate of the model parameters can be adjusted according to the nature of the 
sound field condition in each frequency band. The resulting signals are the estimates of 
30 the clean speech and noise signals. 



When the noise source is not in the direct sound field (reverberant environment) the 
mixing transfer fimctions become compUcated and it is not possible to estimate them in 
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real time on a typical processor of a hearing aid system. However, imder the assumption 
that the speech source is in the direct sound field, the two channel of the binaural system 
always carry information about the spatial position of the speech source and it can be 
used to enhance the signal. A statistical based weighting approach can be used to extract 
5 the speech from the noise. The short-time coherence function allows estimating a 
probabiUty of presence of speech. Such a measure defines a weighting function in the 
time-frequency domain. Applying it to the noisy speech signals allows the determination 
of the regions where speech is dominant and to attenuate regions where noise is 
dominant. 

10 

As it was presented previously, two enhancement approaches are used in the proposed 
approach. The aim of the sound field diffuseness detection is to detect the acoustical 
conditions wherein the hearing aid system is working. The detection block gives an 
indication about the diffuseness of the noise source. The result may be that the noise 

15 source is in the direct soimd field, in the diffuse sound field or in-between. The 
information is given for each Bark or Mel frequency band. The coherence function 
presented previously estimates a measure of diffuseness. When the coherence is equal (or 
nearly equal) to one during speech pauses, the noise source is in the direct sound field. 
When it is close to zero, the noise source is in the diffuse sound field. For intermediate 

20 values, the acoustical environment is between direct and diffuse sound field. 

Once the diffuseness of the sound field is known, the results of the parametric approach 
(source separation) and of the non-parametric approach (coherence) can be combined 
optimally to enhance the speech signals. The combination may be achieved gradually by 
25 weighing the signal provided by source separation through the diffuseness measure and 
the signal provided by the coherence by the complementary value of the diffuseness 
measure to one. 

As the de-mixing transfer functions have been identified during the source separation, 
30 they can be used to reconstruct the spatiality of the soimd sources. The noise source can 
be added to the enhanced speech signal, keeping its directivity but with reduced level. 
Such an approach offers the advantage that the intelUgibility of the speech signal is 
increased (by the reduction of the noise level), but the information about noise sources is 
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kept (this can be useful when the noise source is a danger). By keeping the spatial 
information, the comfort of use is also increased. 



